Data-driven prototyping via natural-language-based GUI retrieval

Rapid GUI prototyping has evolved into a widely applied technique in early stages of software development to facilitate the clarification and refinement of requirements. Especially high-fidelity GUI prototyping has shown to enable productive discussions with customers and mitigate potential misunderstandings, however, the benefits of applying high-fidelity GUI prototypes are accompanied by the disadvantage of being expensive and time-consuming in development and requiring experience to create. In this work, we show RaWi, a data-driven GUI prototyping approach that effectively retrieves GUIs for reuse from a large-scale semi-automatically created GUI repository for mobile apps on the basis of Natural Language (NL) searches to facilitate GUI prototyping and improve its productivity by leveraging the vast GUI prototyping knowledge embodied in the repository. Retrieved GUIs can directly be reused and adapted in the graphical editor of RaWi. Moreover, we present a comprehensive evaluation methodology to enable (i) the systematic evaluation of NL-based GUI ranking methods through a novel high-quality gold standard and conduct an in-depth evaluation of traditional IR and state-of-the-art BERT-based models for GUI ranking, and (ii) the assessment of GUI prototyping productivity accompanied by an extensive user study in a practical GUI prototyping environment

Automatic generation of graphical user interface prototypes from unrestricted natural language requirements

High-fidelity GUI prototyping provides a meaningful manner for illustrating the developers’ understanding of the requirements formulated by the customer and can be used for productive discussions and clarification of requirements and expectations. However, high-fidelity prototypes are time-consuming and expensive to develop. Furthermore, the interpretation of requirements expressed in informal natural language is often error-prone due to ambiguities and misunderstandings. In this dissertation project, we will develop a methodology based on Natural Language Processing (NLP) for supporting GUI prototyping by automatically translating Natural Language Requirements (NLR) into a formal Domain-Specific Language (DSL) describing the GUI and its navigational schema. The generated DSL can be further translated into corresponding target platform prototypes and directly provided to the user for inspection. Most related systems stop after generating artifacts, however, we introduce an intelligent and automatic interaction mechanism that allows users to provide natural language feedback on generated prototypes in an iterative fashion, which accordingly will be translated into respective prototype changes

Recherches d'indices de jeune chez la larve de sole, Solea solea (Linnaeus, 1758): approche experimentale et application dans le golfe de Gascogne

SIGLEAvailable from INIST (FR), Document Supply Service, under shelf-number : TD 81112 / INIST-CNRS - Institut de l'Information Scientifique et TechniqueFRFranc

Automated Retrieval of Graphical User Interface Prototypes from Natural Language Requirements

High-fidelity Graphical User Interface (GUI) prototyping represents a suitable approach for allowing to clarify and refine requirements elicitated from customers. In particular, GUI prototypes can facilitate to mitigate and reduce misunderstandings between customers and developers, which may occur due to the ambiguity and vagueness of informal Natural Language (NL). However, employing high-fidelity GUI prototypes is more time-consuming and expensive compared to other simpler GUI prototyping methods. In this work, we propose a system that automatically processes Natural Language Requirements (NLR) and retrieves fitting GUI prototypes from a semi-automatically created large-scale GUI repository for mobile applications. We extract several text segments from the GUI hierarchy data to obtain textual representations for the GUIs. To achieve ad-hoc GUI retrieval from NLR, we adopt multiple Information Retrieval (IR) approaches and Automatic Query Expansion (AQE) techniques. We provide an extensive and systematic evaluation of the applied IR and AQE approaches for their effectiveness in terms of GUI retrieval relevance on a manually annotated dataset of NLR in the form of search queries and User Stories (US). We found that our GUI retrieval performs well in the conducted experiments and discuss the results

GUI2WiRe: Rapid wireframing with a mined and large-scale GUI repository using natural language requirements

High-fidelity Graphical User Interface (GUI) prototyping is a well-established and suitable method for enabling fruitful discussions, clarification and refinement of requirements formulated by customers. GUI prototypes can help to reduce misunderstandings between customers and developers, which may occur due to the ambiguity comprised in informal Natural Language (NL). However, a disadvantage of employing high-fidelity GUI prototypes is their time-consuming and expensive development. Common GUI prototyping tools are based on combining individual GUI components or manually crafted templates. In this work, we present GUI2WiRe, a tool that enables users to retrieve GUI prototypes from a semi-automatically created large-scale GUI repository for mobile applications matching user requirements specified in Natural Language (NLR). We extract multiple text segments from the GUI hierarchy data and employ various Information Retrieval (IR) models and Automatic Query Expansion (AQE) techniques to achieve ad-hoc GUI retrieval from NLR. Retrieved GUI prototypes mined from applications can be inserted in the graphical editor of GUI2WiRe to rapidly create wireframes. GUI components are extracted automatically from the GUI screenshots and basic editing functionality is provided to the user. Finally, a preview of the application is created from the wireframe to allow interactive exploration of the current design. We evaluated the applied IR and AQE approaches for their effectiveness in terms of GUI retrieval relevance on a manually annotated collection of NLR and discuss our planned user studies